Skip to content

Add blog post on chatbot eval setup#3318

Merged
annabellscha merged 42 commits into
mainfrom
annabell/add-blog-post-on-demo-project
Jul 22, 2026
Merged

Add blog post on chatbot eval setup#3318
annabellscha merged 42 commits into
mainfrom
annabell/add-blog-post-on-demo-project

Conversation

@annabellscha

@annabellscha annabellscha commented Jul 16, 2026 •
edited by greptile-apps Bot
Loading

Copy link
Copy Markdown
Contributor

Summary

  • Add a new blog post, "Yes, you can steal our eval setup"
  • Link to the public Langfuse demo project and related docs
  • Reuse existing blog images for monitoring, annotation queues, and failure modes

Checks

  • pnpm run format
  • pnpm run format:check
  • node scripts/check-h1-headings.js

Note: local pnpm commands printed the existing Node engine warning because the shell is on Node 24 while the repo requests Node 22.

Greptile Summary

This PR adds a blog post about the docs chatbot evaluation setup. The main changes are:

  • New post with public demo links and evaluation guidance.
  • New screenshots for traces, monitors, datasets, and annotation queues.
  • An explicit anchor for the academy dataset-item heading.
  • A checked-in fallback for GitHub star update failures.

Confidence Score: 4/5

The GitHub-stars failure path needs a contained fix before merging.

  • Network failures can safely use the checked-in value.
  • Local file-write failures now follow the same fallback and let the build succeed with stale output.
  • The new MDX content follows established component, frontmatter, heading, and image conventions.

scripts/update-github-stars.js

Prompt To Fix All With AI 
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
scripts/update-github-stars.js:63-67
**Write Failures Exit Successfully**

The catch block now treats every exception as an unavailable GitHub API. If `fs.writeFileSync` fails because the generated file is read-only or the disk is full, the checked-in value makes this script return success, so the prebuild continues and deploys stale data instead of reporting the local build failure. Limit the fallback to fetch and response failures, while allowing file-write failures to stop the build.

Reviews (1): Last reviewed commit: "Merge branch 'main' into annabell/add-bl..." | Re-trigger Greptile

Greptile also left 1 inline comment on this PR.

Context used:

  • Rule used - Move imports to the top of the module instead of p... (source)

Learned From
langfuse/langfuse-python#1387

@vercel

vercel Bot commented Jul 16, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
langfuse-docs Ready Ready Preview, Comment Jul 22, 2026 7:20am

Request Review

@annabellscha
annabellscha removed the request for review from marcklingen July 22, 2026 06:40
@annabellscha
annabellscha marked this pull request as ready for review July 22, 2026 06:40
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

@claude review

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. documentation Improvements or additions to documentation labels Jul 22, 2026
greptile-apps[bot]
greptile-apps Bot reviewed Jul 22, 2026
View reviewed changes
Comment thread scripts/update-github-stars.js Outdated
@annabellscha
annabellscha force-pushed the annabell/add-blog-post-on-demo-project branch from f9990ab to 3165229 Compare July 22, 2026 07:09
@annabellscha
annabellscha added this pull request to the merge queue Jul 22, 2026
@dosubot dosubot Bot added the auto-merge This PR is set to be merged label Jul 22, 2026
@annabellscha
annabellscha removed this pull request from the merge queue due to a manual request Jul 22, 2026
@annabellscha
annabellscha added this pull request to the merge queue Jul 22, 2026
@dosubot dosubot Bot removed the auto-merge This PR is set to be merged label Jul 22, 2026
@annabellscha
annabellscha removed this pull request from the merge queue due to a manual request Jul 22, 2026
@annabellscha
annabellscha added this pull request to the merge queue Jul 22, 2026
@dosubot dosubot Bot added the auto-merge This PR is set to be merged label Jul 22, 2026
Merged via the queue into main with commit c664ad3 Jul 22, 2026
16 checks passed
@annabellscha
annabellscha deleted the annabell/add-blog-post-on-demo-project branch July 22, 2026 08:39
@dosubot dosubot Bot removed the auto-merge This PR is set to be merged label Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Lotte-Verheyden Lotte-Verheyden Lotte-Verheyden left review comments
+1 more reviewer
@greptile-apps greptile-apps[bot] greptile-apps[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

documentation Improvements or additions to documentation size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@annabellscha @Lotte-Verheyden